Various forms of network-based storage systems are known today. These forms include network attached storage (NAS), storage area networks (SANs), and others. Network storage systems are commonly used for a variety of purposes, such as providing multiple users with access to shared data, backing up critical data (e.g., by data mirroring), etc.
A network-based storage system typically includes at least one storage server, which is a processing system configured to store and retrieve data on behalf of one or more client processing systems (“clients”). In the context of NAS, a storage server may be a file server, which is sometimes called a “filer”. A filer operates on behalf of one or more clients to store and manage shared files. The files may be stored in a storage subsystem that includes one or more arrays of mass storage devices, such as magnetic or optical disks or tapes, by using RAID (Redundant Array of Inexpensive Disks). Hence, the mass storage devices in each array may be organized into one or more separate RAID groups.
In a SAN context, a storage server provides clients with block-level access to stored data, rather than file-level access. Some storage servers are capable of providing clients with both file-level access and block-level access, such as certain Filers made by Network Appliance, Inc. (NetApp®) of Sunnyvale, Calif.
In file servers, data is stored in logical containers called volumes, which may be identical with, or subsets of, aggregates. An “aggregate” is a logical container for a pool of storage, combining one or more physical mass storage devices (e.g., disks) or parts thereof into a single logical storage object, which contains or provides storage for one or more other logical datasets at a higher level of abstraction (e.g., volumes). A “volume” is a set of stored data associated with a collection of mass storage devices, such as disks, which obtains its storage from (i.e., is contained within, and may be coextensive with) an aggregate, and which is managed as an independent administrative unit, such as a complete file system. A “file system” is an independently managed, self-contained, hierarchal set of data units (e.g., files, blocks or LUNs). Although a volume or file system (as those terms are used herein) may store data in the form of files, that is not necessarily the case. That is, a volume or file system may store data in the form of other units, such as blocks or LUNs.
One feature, which is useful to have in a storage server is the ability to create a read-only, persistent, point-in-time image (RPPI) of a dataset, such as a volume or a LUN, including its metadata. This capability allows the exact state of the dataset to be restored from the RPPI in the event of, for example, data corruption or accidental data deletion. The ability to restore data from an RPPI provides administrators with a simple mechanism to revert the state of their data to a known previous point in time as captured by the RPPI. Typically, creation of an RPPI or restoration from an RPPI can be controlled from a client-side software tool. An example of an implementation of an RPPI is a Snapshot™ generated by SnapDrive™ or SnapManager® for Microsoft® Exchange software, both made by Network Appliance, Inc. of Sunnyvale, Calif. Unlike other RPPI implementations, NetApp Snapshots do not require duplication of data blocks in the active file system, because a Snapshot can include pointers to data blocks in the active file system for any blocks that have not been modified since the Snapshot was created. The term “Snapshot” is used in this document without derogation of Network Appliance, Inc.'s trademark rights. The “active” file system is a file system to which data can be both written and read. RPPI, in contrast, is a read-only copy of the file system saved at a specific time.
An example of an RPPI technique, which does not require duplication of data blocks to create an RPPI, is described in U.S. Pat. No. 5,819,292, which is incorporated herein by reference and which is assigned to Network Appliance, Inc. of Sunnyvale, Calif. In addition, this technique allows an RPPI to be created quickly, helps to reduce consumption of storage space due to RPPIs, and reduces the need to repeatedly update data block pointers as required in some prior art RPPI techniques.
In order to improve reliability and facilitate disaster recovery in the event of a failure of a storage server, its associated disks or some portion of the storage infrastructure, it is common to “mirror” or replicate some or all of the underlying data and/or the file system that organizes the data. A mirror is a copy of a dataset. A dataset is a set of data. For example, a dataset may be a file system, a volume, a LUN, a sub-volume, etc. In one example, a mirror is established and stored at a remote site. Storing mirror at a remote site increases the likelihood of data recovery in the event of a true disaster that may physically damage the main storage location or its infrastructure (e.g., in the event of flood, power outage, act of war, etc.). The mirror is updated at regular intervals, typically set by an administrator, in an effort to capture the most recent changes to the storage system.
As shown in FIG. 1, a storage system 102 may serve as a backup server. A backup server is a processing system configured to store copies of data from other processing systems. A source dataset is a dataset to be backed up. Backing up of a dataset on a primary storage system 101 (hereinafter “primary server”) starts with the step of copying the dataset on the primary server 101 to the backup server 102. This initial, or baseline, transfer may take some time to complete, as it is duplicating the entire source dataset to the backup server 102. When the initial full backup is performed, the backup server 102 creates an RPPI of the backup copy of the source dataset. Each subsequent backup transfers those data blocks that have changed since the previous backup process. The backup server 102 receives these data blocks, updates its backup copy, and creates a new RPPI of the backup copy. This backup mechanism is called a block-level incremental backup process. An example of such a backup system is the SnapVault® system provided by Network Appliance.
As described above, RPPI technology provides onsite data protection for a dataset in a storage system. Especially with NetApp's Snapshot technology, a number of RPPIs may be created, yet without consuming too much storage space and time. As used herein, providing continuous data protection (CDP) for a dataset means backing up the dataset whenever any change is made to the dataset. As used herein, providing long-term data protection (LTDP) for a dataset means keeping a backup copy of the dataset available for a sufficiently long period of time. A sufficiently long period of time may be, for example, one year or several years. RPPI technology may provide a method of CDP for a dataset in a storage system. However, the number of RPPIs a storage server can maintain is limited by design or physical considerations (e.g., available storage space). Thus, in order to ensure the CDP for the dataset, some RPPIs typically need to be deleted. As a result, some historical backups are lost. Therefore, RPPI technology alone does not provide long-term data protection for the dataset.
On the other hand, block-level incremental backup may provide long-term data protection for a dataset in a storage system. The block-level incremental backup technique, however, is not suitable for CDP, because the backup process consumes too many resources, such as time and network bandwidth.